Feature/multiserver plugin by Muddyblack · Pull Request #3421 · ipspace/netlab

Muddyblack · 2026-05-20T17:30:52Z

Reference: #3420

Summary

This PR adds the multiserver plugin to distribute a single Netlab topology across multiple physical servers.
Sadly for now containerlab-provider only.

Key Details

Self-contained: The implementation is entirely within netsim/extra/multiserver/ and doesn't modify any core Netlab engine logic.
Consistent Allocations: IP, interface, and VNI allocations are computed once on the workstation. The plugin then generates a per-server directory with a filtered clab.yml and netlab.snapshot.pickle.
Native Remote Deployments: Remote servers launch using standard sudo netlab up --snapshot -vv without needing custom CLI options.

For the test-files I am not sure if they make any sense. But they show at least it does not interfere with the normal netlab workflow.

Explanations on how it works can be found in docs/plugins/multiserver.md.

ipspace · 2026-05-24T07:29:45Z

+
+```{warning}
+* The *multiserver* plugin requires the **containerlab** provider on all servers.
+* Containerlab version >= `0.46` is required for native VXLAN link endpoint support.


Just a thought: the plugin probably does not work with old releases, and we're enforcing 0.75.0 right now.

What do you mean?

Well I only tried it on 0.75 true but it should work before too. It only requires a netlab version which uses the pickel system for its storage.

Netlab already enforces a higher version than 0.46 i think no?

Netlab already enforces a higher version than 0.46 i think no?

That was the point. You don't have to mention containerlab version.

ipspace

Super-awesome-job!!! Thanks a million.

Tons of comments (as you expected ;). Some of them are just suggestions or pointers to existing helper functions, in other cases I think we can make the whole thing a lot more streamlined with significant rewrites.

ipspace · 2026-05-24T07:32:12Z

+(multiserver-servers)=
+### Server Parameters
+
+Each entry in the **multiserver.servers** list supports these parameters:


Would it make more sense to have a dictionary of servers?

ipspace · 2026-05-24T07:36:00Z

+
+| Parameter | Type | Meaning |
+|-----------|------|---------|
+| **id** | integer | Unique identifier for the server (e.g. `1`, `2`) |


ID could be assigned automatically, like we do for nodes. We even have a set of functions (modules/_dataplane) to handle IDs where some objects have a static ID and others need auto-assigned ones -- used for VLANs, VRFs, and the like

ipspace · 2026-05-24T07:37:27Z

+In `auto` mode, nodes that are not explicitly pinned to a server are distributed automatically using a greedy balancing algorithm:
+
+1. Nodes belonging to a *netlab* group are kept together — the entire group is placed on the server that currently has the fewest nodes. Larger groups are placed first for better balance.
+2. Remaining ungrouped nodes are assigned one at a time to the least-loaded server.


Haven't looked at the code yet, but I'm guessing that "loaded" means "number of nodes", not something more complex like CPU/RAM requirements of lab devices? If that's the case, it might be worth spelling it out.

ipspace · 2026-05-24T07:38:51Z

+
+### Automatic Assignment
+
+In `auto` mode, nodes that are not explicitly pinned to a server are distributed automatically using a greedy balancing algorithm:


In some future version, you might want to add server capabilities for weighted distribution ;)

ipspace · 2026-05-24T07:50:27Z

+
+The plugin automatically copies all required files into each server directory — no extra bundling step is needed.
+
+**Step 2: Copy server directories to remote hosts** (e.g. via rsync):


We might want to automate that in the future, but this is definitely more than good enough for version 1

ipspace · 2026-05-26T09:05:42Z

+
+def _intf_clab_name(intf: Box) -> str:
+  """Containerlab interface name for a node interface."""
+  return intf.get("clab", {}).get("name", "") or intf.get("ifname", "")


intf.get("clab.name","") or intf.get("ifname","")

See "Using Box objects" in AGENTS.md

ipspace · 2026-05-26T09:07:56Z

+  return intf.get("clab", {}).get("name", "") or intf.get("ifname", "")
+
+
+def _build_clab_node(nname: str, ndata: Box, topology: Box) -> dict:


It looks like this whole part could be easier to do with a topology filter (something similar to "remove unmanaged nodes") followed by an augmented clab.yml Jinja2 template. I also wouldn't have a problem adding VXLAN support (as clab attributes) into netlab core and then using those attributes here.

ipspace · 2026-05-26T09:08:42Z

+
+# ===========================================================================
+#  Internal helpers — clab.yml generation
+# ===========================================================================


This whole section should be completely restructured. We're reinventing the wheel (in Python)

ipspace · 2026-05-26T09:11:13Z

+  topo_copy.links = [l for l in topo_copy.links if any(i.node in local_nodes for i in l.get("interfaces", []))]
+
+  # Expand paths (add f_files / f_tasks / f_dirs computed keys).
+  make_paths_absolute(topo_copy.defaults.paths)


I'm afraid there are lots of assumptions here, starting with "the directory structure MUST be the same on all the servers and the control node"

ipspace · 2026-05-26T09:11:43Z

+    pickle.dump(topodict, f)
+
+
+def _write_vxlan_scripts(out_dir: str, tunnels: list, dev: str) -> None:


Any reason why you wouldn't use a Jinja2 template for this?

Muddyblack added 4 commits May 20, 2026 16:46

init multiserver plugin

cac59e3

explain interface overriding

c0c2a1a

divide and conquer

cdb71d5

rm --- from yml

c9f2a6f

Muddyblack linked an issue May 20, 2026 that may be closed by this pull request

Proposal: multiserver plugin to easily split topologies across physical servers #3420

Open

ipspace reviewed May 24, 2026

View reviewed changes

ipspace requested changes May 26, 2026

View reviewed changes


		### Automatic Assignment

		In `auto` mode, nodes that are not explicitly pinned to a server are distributed automatically using a greedy balancing algorithm:


		The plugin automatically copies all required files into each server directory — no extra bundling step is needed.

		Step 2: Copy server directories to remote hosts (e.g. via rsync):

		return intf.get("clab", {}).get("name", "") or intf.get("ifname", "")


		def _build_clab_node(nname: str, ndata: Box, topology: Box) -> dict:

		pickle.dump(topodict, f)


		def _write_vxlan_scripts(out_dir: str, tunnels: list, dev: str) -> None:

Conversation

Muddyblack commented May 20, 2026

Summary

Key Details

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Muddyblack May 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ipspace left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Muddyblack May 25, 2026 •

edited

Loading